Notes on Motion Estimation

There are a great variety of applications that depend on analyzing the motion in image sequences. These include motion detection for surveillance, image sequence data compression (MPEG), image understanding (motion-based segmentation, depth/structure from motion), obstacle avoidance, image registration and compositing. The rst step in processing image sequences is typically image velocity estimation. The result is called the optical ow eld, a collection of two-dimensional velocity vectors, one for each small region (potentially, one for each pixel) of the image. Image velocities can be measured using correlation or block-matching (for example, see Anandan, 1989) in which each small patch of the image at one time is compared with nearby patches in the next frame. Feature extraction and matching is another way to measure the ow eld (for reviews of feature tracking methods see Barron, 1984, or Aggarwal and Nandhakumar, 1988). Gradient-based algorithms are a third approach to measuring ow elds (for example, Horn and Schunk, 1981; Lucas and Kanade, 1981; Nagel, 1987). A fourth approach using spatiotemporal ltering has also been proposed (for example, Watson and Ahumada, 1985; Heeger, 1987; Grzywacz and Yuille, 1990; Fleet and Jepson, 1990). This handout concentrates on the lter-based and gradient-based methods. Emphasis is placed on the importance of multiscale, coarse-tone, re nement of the velocity estimates. For a good overview and comparison of di erent ow methods, see (Barron et al, 1994).